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DETAILED ACTION 

1 . This office action is responsive to communication filed on February 4, 2008. 
Claims 16 and 17 are pending in the application and have been examined by the 
Examiner. 

Response to Arguments 

2. Applicant's arguments with respect to claim 16 have been considered but are 
moot in view of the new ground(s) of rejection. 

Claim Objections 

3. Claim 17 is objected to because of the following informalities: Lack of clarity and 
precision. 

Claims 17 recites, "the apparatus according to claim 11". However, claim 11 has 
been cancelled by Applicant. Upon further examination, it appears that claim 17 is 
meant to depend from claim 16. Therefore, the Examiner will interpret claim 17 to read, 
"the apparatus according to claim 16". Appropriate correction is required. 

Claim Rejections - 35 USC § 103 

4. The following is a quotation of 35 U.S.C. 1 03(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

5. The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1 , 148 
USPQ 459 (1966), that are applied for establishing a background for determining 
obviousness under 35 U.S.C. 103(a) are summarized as follows: 
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1 . Determining the scope and contents of the prior art. 

2. Ascertaining the differences between the prior art and the claims at issue. 

3. Resolving the level of ordinary skill in the pertinent art. 

4. Considering objective evidence present in the application indicating 
obviousness or nonobviousness. 

6. Claim 16 is rejected under 35 U.S.C. 103(a) as being unpatentable over 

Nakamura et al. (US 6,963,374) in view of Anderson (US 6,847,388) and Taniguchi et 

al. (US 5,619,347). 

Consider claim 16, Nakamura et al. teaches: 

An image sensing apparatus ("Digital Camera", figures 1-4, column 2, line 56 
through column 4, line 43) comprising: 

an image sensing device ("CCD", 303, figure 4) which outputs image data 
obtained by an image sensing element as RAW data (column 3, lines 50-58); 

a memory ("DRAM", 232, figure 4) which has a first area for temporally storing 
first RAW data obtained in a first image sensing operation of said image sensing device 
and for temporally storing second RAW data obtained in a second image sensing 
operation next to the first image sensing operation of said image sensing device (See 
figure 7, column 7, lines 10-49. First raw image data is written into the DRAM (232) 
over channel 1 . During a subsequent image capture operation, this first raw data is 
read out of DRAM (232) over channel 2 for image processing, as second image data is 
written into the DRAM (232) over channel 1 .); 

a white balance integration device (21 1a, figure 6A) which integrates at least one 
of the first and second RAW data for a white balance processing (The white balance 
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integration device (21 1 a) is part of the image signal processor (21 1 , figure 4). The 
RAW image data is subjected to white balance processing, and then stored into the 
DRAM (232), column 6, lines 21-26.); 

an image processing device (21 1 , figure 4) which performs image processing of 
the first and second RAW data readout from said memory (The image processing 
device (21 1) performs processing such as color space conversion on the RAW data 
readout from memory (232), column 4, lines 8-10, column 6, lines 26-36, column 7, lines 
24-33.), 

a display device ("EVF", 20, or "LCD", 10, figure 4) which displays an object 
image during imaging on the image sensing element (The display acts as a "live view 
display" (i.e. an object image is displayed during imaging), column 3, lines 16-23.); and 

a control device ("main CPU", 21, figure 4) which controls said memory (232), 
said white balance integration device (211a), said image processing device (211), and 
said display device (20, 10, column 4, lines 1-27. The main CPU (21) comprises the 
image processing device (21 1 ) which contains the white balance integration device 
(21 1a), a bus controller (218) for controlling the memory (232), and a video encoder 
(213) which supplies analog image signals to the display.), 

wherein, in case said image sensing device outputs third RAW data obtained in a 
third image sensing operation next to the second image sensing operation (See figure 
7. Steps 1, 8 and 10-12 comprise a continuous loop. Because of this, it is clear that 
third RAW data, fourth RAW data, etc. can be output by the image sensing device 
without altering the camera operation.), said control device (21) controls so that, said 
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image processing device (211) processes a color space conversion for the first RAW 
data readout from said memory (232) in accordance with start of exposure/storage of 
the second RAW data from the image sensing element (303) in the second image 
sensing operation (See figure 8, column 7, lines 24-33. A frame of raw data obtained by 
an immediately preceding image (i.e. first image data) is read out of DRAM (232) and 
subjected to color space conversion while second image data is obtained through 
exposure and storage of the CCD.), and said display device ("LCD", 10) displays the 
object image (See "Live View Display" on the right side of figure 8.) after the color space 
conversion processing for the first RAW data (See "Pc", figure 8) and the integral 
processing (See Readout 2, "Pe", figure 8) for the second RAW data (See column 7, 
lines 24-49. The LCD exhibits a "Live View Display" after the color space conversion 
"Pc" and integral processing "Pe".). 

However, Nakamura et al. does not explicitly teach that said memory (232) has a 
second area for storing the second RAW image data, or that said memory stores the 
third RAW data in the first area in which the first RAW data after processing of the first 
RAW data by said image processing device is finished. Nakamura et al. further does 
not explicitly teach that the processing device processes a color space conversion for 
the first RAW data readout from said first area in accordance with start of reading the 
second RAW data from the image sensing element. 

Anderson is similar to Nakamura et al. in that Anderson teaches of a camera 
(figures 1-3) with a memory (figure 4a). Anderson also similarly teaches of reading out 
raw image data from an image sensor (1 14, figure 1 , column 5, lines 59-64), storing the 
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data in a memory (530, column 5, line 59 through column 6, line 3), and subsequently 
performing color space conversion on the image data (column 8, line 59 through column 
9, line 7). 

However, in addition to Nakamura et a!., Anderson teaches that said memory 
(figures 4a and 4b) has a second area (Input buffer, 2, B) for storing the second RAW 
image data, that said memory stores the third RAW data in the first area in which the 
first RAW data after processing of the first RAW data by said image processing device 
is finished, and that the processing device processes a color space conversion for the 
first RAW data readout from said first area in accordance with start of reading the 
second RAW data from the image sensing element (See figure 4a, column 4, line 59 
through column 6, line 3, column 6, lines 38-56, column 8, line 59 through column 9, 
line 8. Anderson teaches, "Referring again to FIG. 4B, the ping-pong buffers are 
utilized during live view mode as follows. While input buffer A is filled with image data, 
the data from input buffer B is processed and transmitted to frame buffer B. At the 
same time, previously processed data in frame buffer A is output to the LCD screen 
402 for display. While input buffer B is filled with image data, the data from input buffer 
A is processed and transmitted to frame buffer A. At the same time, previously 
processed data in frame buffer B is output to the LCD screen 402 for display." As one 
buffer (i.e. the second area) is filled with raw image data (i.e. the start of reading of the 
second RAW data from the image sensing element), the other buffer (i.e. the first area) 
is emptied and processed (i.e. first RAW data is readout from the first area), which 
processing involves color space conversion (See 612, figure 7, column 8, line 59 
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through column 9, line 8). Anderson teaches that the input buffers A and B alternate 
between an input cycle and a processing cycle, column 6, lines 8-10. Therefore, buffer 
A is an input buffer during a first phase, an output buffer during a second phase when 
second RAW data is written into buffer B, and again an input buffer during a third 
phase when third RAW data is output from the sensor an into memory. Thus, the third 
image data is stored in the same area (i.e. buffer A) as the first image data during the 
third phase, after the first image data is processed during the second phase.). 

Therefore, it would have been obvious to a person having ordinary skill in the art 
at the time of the invention to use ping-pong buffers as taught by Anderson in the 
camera taught by Nakamura et al. to read out raw image data from the image sensor 
concurrent with the processing color space conversion of previous image data, for the 
benefit of improving the display speed of the digital camera and preventing the tearing 
of the image on the display (Anderson, column 5, line 65 through column 6, line 3.). 

Nakamura et al. alone does not explicitly teach that the integral processing 
for the second RAW data by said white balance integration device and the color 
space conversion for first RAW data by said image processing device processes 
are performed in parallel during reading of the second RAW data from the image 
sensing element. However, because the white balance integration taught by 
Nakamura et al. is performed during the raw data writing ("Pe"), and thus in 
parallel with the readout of the second image data (see figure 8, column 7, lines 
41-49), the combination of Nakamura et al. and Anderson teaches that that 
integral processing of the second image data and the color space conversion of 
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the first image data are performed in parallel. This is because Anderson modifies 
Nakamura et al. such that the image processing of the first image data (which 
includes color space conversion) readout from the first area of memory takes 
place concurrently with the readout of second RAW image data from the image 
sensor (which includes integration) and into the second area of memory, as 
discussed above. 

However, the combination of Nakamura et al. and Anderson does not explicitly 
teach a white balance calculation device which calculates a white balance coefficient on 
the basis of the integration result by the white balance calculation, or that the image 
processing is performed in accordance with the white balance coefficient calculated by 
said white balance calculation device. 

Taniguchi et al. is similar to Nakamura et al. in that Taniguchi et al. teaches 
performing white balance (column 1 , lines 9-15) on image data stored in a picture 
memory (12, figure 1 , column 9, lines 42-60). Taniguchi et al. also similarly teaches of a 
white balance integration device (15, 16, figure 1, column 9, line 63 through column 10, 
line 8). 

However, in addition to the teachings of Nakamura et al. and Anderson, 
Taniguchi et al. teaches of a white balance calculation device ("white balance coefficient 
calculating unit", 22, figure 1) which calculates a white balance coefficient on the basis 
of the integration result by the white balance calculation (See figure 1, column 10, lines 
45-60. The white balance calculation device (22) calculates a white balance coefficient 
according to a plurality of factors determined in units 17-21 of figure 1, based upon the 
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white balance integration of the white balance integration device (15,16). See also 
column 10, lines 6-44.), and that the image processing is performed in accordance with 
the white balance coefficient calculated by said white balance calculation device (See 
14, figure 1, column 9, lines 50-60, column 10, lines 56-60. Image processing is 
performed by the white balance adjusting unit (14) based upon the white balance 
coefficient calculated by the white balance coefficient calculating unit (22).). 

Therefore, it would have been obvious to a person having ordinary skill in the art 
at the time of the invention to have the image processing device taught by the 
combination of Nakamura et al. and Anderson comprise a white balance calculation 
device and perform white balance processing based upon a calculated white balance 
coefficient as taught by Taniguchi et al. for the benefit of performing a sufficient degree 
of white balance adjustment appropriate to a colored picture without any erroneous 
adjustment or adverse influence due to a high chromaticity region of the colored picture 
(Taniguchi et al., column 2, lines 40-50). 

The combination of Nakamura et al., Anderson, and Taniguchi et al. teaches that 
said white balance calculation device (taught by Taniguchi et al.) calculates the white 
balance coefficient (see above rationale) while said display device displays the object 
image after the parallel processing. Anderson teaches that parallel processing is 
performed by utilizing two buffers, column 6, lines 47-56. Anderson teaches that the 
image data from one buffer is displayed on the LCD (402) as the image data from the 
other buffer is processed, column 6, lines 47-56. Therefore, an object image is 
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continuously displayed on the display, including any time when a white balance 
coefficient is being calculated. 

7. Claim 17 is rejected under 35 U.S.C. 103(a) as being unpatentable over 
Nakamura et al. (US 6,963,374) in view of Anderson (US 6,847,388) and Taniguchi et 
al. (US 5,619,347) as applied to claim 16 above, and further in view of Sasaki (US 
6,961,085). 

Consider claim 17, and as applied to claim 16 above, the combination of 
Nakamura et al., Anderson, and Taniguchi et al. teaches that said display device 
displays the object image after said white balance calculation device calculates the 
white balance coefficient (See claim 16 rationale. Anderson teaches of a continuous 
display.). However, the combination of Nakamura et al., Anderson, and Taniguchi et al. 
does not explicitly teach a defect correction device. 

Sasaki is similar to Nakamura et al. in that image data is collected from the 
image sensor (12, figure 1 ), preliminary processing is performed to yield first image data 
(see step 41 , figure 7) which is written into a buffer memory (26a, figure 7). Sasaki also 
similarly teaches that the first image data is read from the buffer memory (step 42, 
figure 7) for additional processing (see column 10, lines 1-57). 

However, in addition to the combined teachings of Nakamura et al., Anderson, 
and Taniguchi et al., Sasaki teaches that the apparatus further comprises a defect 
correction device which corrects a defective pixel portion of image data when the image 
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sensing element has a defective pixel (See column 10, lines 21-57. Sasaki teaches that 
the locations of defective pixels are stored in memory, and when reading the data 
output from the buffer (26a), the defective pixels are corrected.), and that a control 
device controls said defect correction device in such a way that said defect correction 
device corrects a defective pixel portion of the image data (See column 10, lines 29-31, 
lines 37-41.). 

Therefore, it would have been obvious to a person having ordinary skill in the art 
at the time of the invention to have a defect correction device for correcting defective 
pixels as taught by Sasaki correct defective pixels in the image processing device 
during the display of the object image as taught by the combination of Nakamura et al., 
Anderson and Taniguchi et al. for the benefit of keeping the influence of a defective 
pixel to a minimum and preserving a high-definition image (Sasaki, column 3, lines 1-5, 
lines 12-16). 

Conclusion 

8. Applicant's amendment necessitated the new ground(s) of rejection presented in 
this Office action. Accordingly, THIS ACTION IS MADE FINAL. See M PEP 
§ 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 
CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 



Application/Control Number: 10/702,200 Page 12 

Art Unit: 2622 

shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1 .136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the date of this final action. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to ALBERT H. CUTLER whose telephone number is 
(571)270-1460. The examiner can normally be reached on Mon-Thu (9:00-5:00). 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Tuan V Ho can be reached on (571 )-272-7365. The fax phone number for 
the organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 



AC 

05/23/2008 



/Ngoc-Yen T. VU/ 
Supervisory Patent Examiner, Art Unit 2622 
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